fix: caller-controlled scope IDs grow Runtime caches without bound - #1325
Merged
Conversation
Signed-off-by: ThunGuo <tew@apache.org>
Contributor
There was a problem hiding this comment.
Pull request overview
This PR introduces a bounded, lease-aware LRU cache for scope-local built-in Runtime resources to prevent unbounded memory growth when callers use many one-off scope_id values (especially via long-running HTTP/MCP server processes). It also adds configuration, metrics, documentation, and regression tests around the new behavior.
Changes:
- Add a lease-aware
ScopeCacheto bound inactive scope-local Runtime resources (default 128) and evict corresponding relational/provider scope resources together. - Add
RuntimeConfig.scope_cache_size(and env-based configuration via existing nested settings) plus/metricsreporting for active/cached scope counts without usingscope_idlabels. - Add regression tests covering one-off scope IDs, convergence to capacity, lock waiter/holder safety, and environment-driven configuration.
Reviewed changes
Copilot reviewed 12 out of 12 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/test_server.py | Verifies server settings load runtime.scope_cache_size from environment. |
| tests/test_server_metrics.py | Adds regression test ensuring one-off scope IDs don’t produce unbounded Runtime scope retention and that metrics avoid scope_id labels. |
| tests/builtin/runtime/test_scope_cache.py | Adds targeted concurrency/eviction regression test for the scope lease + lock interaction under cache pressure. |
| src/powercontext/server/metrics.py | Adds powercontext_server_runtime_scopes{state=...} gauge and setter. |
| src/powercontext/server/factory.py | Wires Runtime scope cache observer into server metrics. |
| src/powercontext/builtin/runtime/relational.py | Adds evict() to drop per-scope cached contexts and locks together. |
| src/powercontext/builtin/runtime/config.py | Adds RuntimeConfig.scope_cache_size with default and validation. |
| src/powercontext/builtin/runtime/composition.py | Plumbs scope_cache_size, evictor, and observer into BuiltinRuntime composition. |
| src/powercontext/builtin/runtime/application.py | Integrates ScopeCache, ensures Runtime close clears scope cache, and refines operation lifecycle tracking for nested operations. |
| src/powercontext/builtin/runtime/_scope_cache.py | New lease-aware, bounded LRU scope cache implementation with eviction hooks and observer support. |
| docs/en/docs/reference/configuration.md | Documents POWERCONTEXT_SERVER_RUNTIME_SCOPE_CACHE_SIZE. |
| docs/zh/docs/reference/configuration.md | Documents POWERCONTEXT_SERVER_RUNTIME_SCOPE_CACHE_SIZE. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
thunguo
marked this pull request as draft
August 23, 2026 10:26
AlexStocks
reviewed
Aug 23, 2026
AlexStocks
reviewed
Aug 23, 2026
Signed-off-by: thunguo <tew@apache.org>
thunguo
marked this pull request as ready for review
August 25, 2026 12:03
38 tasks
AlexStocks
approved these changes
Aug 26, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue or RFC does this PR close?
Closes #1322.
Rationale for this change
Caller-controlled
scope_idvalues could permanently populate several process-level Runtime and relational-provider caches. A long-running HTTP or MCP Server therefore exhibited linear memory growth when receiving valid requests with many one-off scope IDs, even when no Source or Memory data was persisted.The Runtime needs a bounded lifecycle for inactive scope-local objects while preserving per-scope serialization for operations that currently hold or are waiting for a lock.
What changes are included in this PR?
RuntimeConfig.scope_cache_sizeand thePOWERCONTEXT_SERVER_RUNTIME_SCOPE_CACHE_SIZEenvironment variable.powercontext_server_runtime_scopes{state="active"}powercontext_server_runtime_scopes{state="cached"}scope_idas a label.Are there any user-facing changes?
Yes, all changes are additive:
RuntimeConfig.scope_cache_sizedefaults to128.POWERCONTEXT_SERVER_RUNTIME_SCOPE_CACHE_SIZE./metricsnow exposes active and cached scope counts using a boundedstatelabel.There are no breaking HTTP, MCP, OpenAPI, or persisted-data changes. If an inactive scope is evicted, its ephemeral services are recomposed on the next request; persisted scope data is unaffected.
How was this change tested?
Targeted Runtime, provider, Server, and metrics tests:
Result:
67 passed.Repository validation:
Result:
464 passed, 7 skipped, 1 deselected.The deselected inference tracing test was also run independently and fails because the current Pydantic AI instrumentation includes schema-retry content in span attributes. No inference or tracing implementation is changed by this PR.
Manual validation sent 250 successful
POST /v1/context/preparerequests using distinct scope IDs:250.128, with0active scopes after the requests completed.AI usage statement